On the Mathematics of the Free-choice Paradigm

Experimental design can be tricky to get right. A very potent illustration of this point is found in Chen and Risen’s recent identification of a logical flaw in a number of past free choice experiments studying the psychological concept of cognitive dissonance[4]. This illustration is unique in many ways. First, the mistake is subtle yet elementary; in the simplest of the affected experiments, the mistake is equivalent to the error of misunderstanding the Monty Hall problem. Second, the mistake affected a fairly large number of research experiments performed over a span of five decades. Third, it challenged some of the experimental evidence for dissonance theory, which is a well known and celebrated area of social psychology. In fact, the term “cognitive dissonance” has migrated from the scientific realm into popular culture, where it occurs frequently in New York Times articles, is the title of a Pod Cast, is featured in several Dilbert cartoons, and has been used to explain everything from why President Clinton was not impeached to why some weight loss programs work better than others. In this paper, our first goal is expository – we wish to tell this interesting story, which seems to have largely escaped the notice of the mathematics community. We will describe the theory of cognitive dissonance, the experimental evidence on which it rests, and the flaw observed by Chen and Risen. Our second goal is to propose methods by which this mistake can be fixed. Chen and Risen already implemented a modified experiment that used a novel type of control group to avoid the logical pitfall. We will describe experimental designs by which the flaw can be avoided even without a control group. Our methods are based on the inherent symmetry of a free-choice experiment. Our third goal is to describe additional problems with all of these free choice experiments (including our own methods and the ChenRisen method), calling back into question whether any free choice experiment can correctly measure the effects of cognitive dissonance.